Solutions for automatically tracking a person's eyes have been known for many years. Eye tracking is understood as the process of measuring the motion of an eye relative to the head, whereas gaze tracking is the process of determining the point of gaze (i.e. where a subject is looking). Consequently, an eye/gaze tracker is a device for measuring eye positions and/or eye movements. Eye/gaze trackers are used in research on the visual system, in psychology, in cognitive linguistics and for controlling various computer systems. An eye/gaze tracker may also be employed in vehicles to determine driver drowsiness or to enable interaction with interior displays. There are a number of different methods for measuring eye movement. However, the most popular variants use video images from which the eye position is extracted.
It is often challenging to find the subject's eyes in the video data, and thereafter maintain tracking thereof. Normally, an initial set of image data represents a larger view of a scene, and then, after having identified eye candidates, only the region on the imaging sensor corresponding to where the eyes are located is read out. Namely, this decreases the average bandwidth need, as well as increases the image sensor read-out frequency. Hence, the effects of any non-controlled light sources due to shorter read out times can be reduced. This strategy is generally referred to as image cropping, windowing or applying an ROI (region of interest) or an AOI (area of interest). When doing so, the image size becomes small, however typically, in each image, the highest possible resolution that the sensor can provide is used.
A generally efficient method of finding and tracking an object (such as a face or particular features therein) is to use image-based recognition techniques that are based on other algorithms than those being most efficient for gaze tracking. Namely, the objects that are identified and subsequently processed differ significantly in size and shape. Therefore, for feature recognition it is beneficial to use the camera's full field of view. Further, in eye tracking, when using only ROI images there is a risk that the eye tracker “locks in” to an eye candidate which in fact is not an eye. Hence, it may be useful to switch from a wide field of view to an ROI (after having found a subject's eyes), and then switch back to a wide field of view, in case one or more eyes fall outside of the ROI. This is also beneficial in situations where multiple subjects are located in front of the eye tracker.
WO2008/107713 discloses a method and apparatus for image processing for object recognition applications. Here, first, an image is acquired at relatively low resolution of a relatively wide field of view. A pattern recognition algorithm is applied to the first image to identify objects of interest. After having determined at least one object of interest and its coordinates within the image, either the same image capture device or an additional image capture device is controlled so as to obtain a sequence of second, relatively high resolution images of a relatively narrow field of view containing the at least one object of interest.
US 2009/0219387 describes an automatic video surveillance system, wherein, initially, a video camera registers a series of low resolution images. This video data is analyzed, and based thereon, at least one region of interest is determined in a scene acquired by the video camera. The at least one region of interest is tracked at a resolution level that is higher than that of the initial series of images. The higher resolution level is dynamically selected so that important details within a scene receive appropriate scrutiny while uninteresting areas are imaged at a lower resolution. Preferably, the subsequent video data are registered in parallel with different exposure levels to provide a greater dynamic range, and thus improved image quality.
Hence, there are examples of prior-art solutions where image data are initially registered at comparatively low resolution, and subsequently, particularly interesting areas are tracked at higher resolution and/or improved image quality. In the context of this invention, the term “resolution of an image”, “resolution” or other such terms is intended to refer to how large the field of view imaged per pixel is. More specifically, the resolution is how large the imaged field of view is divided by the number of pixels, i.e. the horizontal field of view of the image divided by the image width in pixels, or the vertical field of view that is imaged divided by the image height in pixels. Thus, an image of relatively high resolution has many pixels per degree field of view, whereas an image of relatively low resolution has fewer pixels per degree field of view.
Nevertheless, in order to attain a robust eye/gaze tracking system it is important to repeatedly verify that the “eyes” being tracked indeed correspond to the image elements being the best eye candidates in the full camera field of view.
For gaze tracking it is well known that (given the gaze angle, eye position and corneal radius) the relative position of an illuminator and a camera determines the position of the sharp reflection of said illuminator on the cornea. The reflection, or glint, may sometimes be unfavorably positioned for gaze tracking, for instance by causing reflections in eye glasses, obscuring large portions of the pupil, or coinciding with a region of the cornea, which is poorly represented by the eye model used.
In such situations, on one hand, it would be highly desirable if information from any other illuminators of the system could be used. On the other hand, a system in which illuminators can be selected adaptively would be associated with problems in terms of testability and repeatability because such a system would show a different behavior in different situations. Moreover, any hardware-related problems with respect to stability (e.g. caused by uneven and unpredictable heat distributions) as well as any visible flicker from low frequency components caused by random switching between illuminators can be prevented by having a repeatable behavior of the system.